The present invention generally relates to distributed file systems, and more particularly to object management in distributed file systems.
In a distributed file system the data are distributed between network-connected nodes. Distributed file systems are scaleable to satisfy increasing storage and access requirements. As more storage resources are required to satisfy the requirements of various applications, new nodes with additional storage capacity can be added to the distributed file system.
The structuring of data between different nodes may impact the performance of certain applications, depending on data access patterns. For example, if one node provides access to two data sets, heavy access to the two data sets may impact the required performance level. To alleviate this type of performance bottleneck, contention for the storage resources is reduced by redistributing selected data sets amongst the different nodes.
Data are traditionally migrated at the block level, with no movement of the associated control data. Such control data includes, for example a file system directory entry, which is sometimes referred to as xe2x80x9cmeta-dataxe2x80x9d. If the control data associated with the competing data sets are not separated, the performance level may continue to be negatively impacted. Because referencing data at the block level does not indicate whether the block contains data or meta-data, it is not feasible to migrate meta-data and data blocks together.
Another approach to redistributing data is to move data at the granularity of a file system. This results in both control data and block-level data being moved. However, a file system typically includes many files that are hierarchically organized within many different directories. Thus, moving an entire file system may not separate the data sets that need to be separated. It can be seen with present systems that the granularity of moving data at the block level may be too fine, and the granularity of moving data at the file system level may be too coarse.
Migrating objects in a distributed file system also presents issues for the clients that reference the objects. One issue is the validity of cached file handles after objects have been migrated. Generally, the file handle identifies the host node and other information for referencing a file. The file handle is provided by a namespace server to a client in response to an access request, and the client caches the file handle for future references. If the data referenced by the file handle is moved, the cached file handle will be invalid. Thus, cached file handle must be revalidated for it to be useful to the client.
A system and method that address the aforementioned problems, as well as other related problems, are therefore desirable.
In various embodiments, the invention updates references to a migrated object in a distributed file system. A migrated object is an object has moved from a source partition server to a target partition server. Each object has an associated parent object and each partition server owns a subset of objects of the file system. A set of forward pointers that reference one or more child objects of the parent object is maintained in each parent object. In the migrated object a set of back pointers is maintained that reference one or more parent objects having forward pointers to the migrated object. To each partition server that owns a parent object of the migrated object, an update request is transmitted. The forward pointers in each parent object are updated in response to the update request.
Various example embodiments are set forth in the Detailed Description and claims which follow.